Filesystems organize and track where data is stored in memory and where free or available space exists in memory. In addition to storing and tracking the actual data of a file, filesystems also store and track metadata associated with the file. The metadata provides information about the file, such as when the file was created or modified, access permissions, ownership of the file, etc.
Distributed or clustered filesystems can store thousands or even millions of files. These files and the corresponding metadata are distributed across a large number of storage devices, such as disk drives.
Managing such a large number of files is a difficult task. In order to acquire information about the files and metadata, large directories or trees are searched. Since this information is spread across many disks, these searches can be slow and process intensive.
Methods and systems to improve the efficiency of searching large file systems are needed.